Image Sampling from Multicast Streams

ABSTRACT

The disclosed methods and system may gain access to a plurality of content streams, and periodically capture still images from each stream. The captured stills may be stored in a predetermined location, and the location may be made available to various clients. By discovering the locations, application clients can issue requests to receive the latest image for a current stream, and use that image to improve a user interface or enhance a user experience in consuming the content.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application claims priority to U.S. Provisional ApplicationNo. 61/442,628, filed Feb. 14, 2011, and entitled “Image Sampling FromMulticast Streams,” the contents of which are hereby incorporated byreference as a non-limiting example embodiment.

BACKGROUND

With the increasing proliferation of content, such as video content,being offered to users, there is an increased demand for easy ways tolet users view, search for and/or access the content, or allow users tocommunicate information related to content.

SUMMARY

Some aspects of features described herein relate to helping users locateand use the content that is available to them. In one aspect, a servercan be used to obtain periodic image samples for various video services(e.g., television channels, movies, etc.), and offer those periodicimage samples to client devices and/or applications that, in turn, canuse those samples to help their users locate or use desired content.

In some embodiments, one or more servers may simultaneously access orsubscribe to a plurality of content sources, such as video multicaststream sources. The servers may periodically capture images from eachstream, and store the images in different corresponding storagelocations associated with different streams. The storage locations maythen be provided or made accessible to requesting clients, who mayreceive periodically updated still images from a plurality of live videostreams.

This summary is not intended to identify critical or essential featuresof the disclosures herein, but instead merely summarizes certainfeatures and variations thereof. Other details and features will also bedescribed in the sections that follow.

BRIEF DESCRIPTION OF THE DRAWINGS

Some features herein are illustrated by way of example, and not by wayof limitation, in the figures of the accompanying drawings and in whichlike reference numerals refer to similar elements.

FIG. 1 illustrates an example information distribution network.

FIG. 2 illustrates an example hardware or software platform on which thevarious elements described herein can be implemented.

FIGS. 3 A&B illustrate an example process for managing one or more imagecapture servers.

FIGS. 4 A&B illustrate example screenshots for use by users and clients.

FIG. 5 illustrates another example screenshot for use by users andclients.

FIG. 6 illustrates an example interface with an image capture server.

FIG. 7 illustrates an example mosaic of images that can be arrangedand/or displayed using the management interface shown in FIG. 6.

DETAILED DESCRIPTION

FIG. 1 illustrates an example information distribution network 100 onwhich many of the various features described herein may be implemented.Network 100 may be any type of information distribution network, such assatellite, telephone, cellular, wireless, etc. One example may be awireless network, an optical fiber network, a coaxial cable network or ahybrid fiber/coax (HFC) distribution network. Such networks 100 use aseries of interconnected communication lines 101 (e.g., coaxial cables,optical fibers, wireless links, etc.) to connect multiple homes 102 orother user locations to a central office or headend 103. The centraloffice 103 may transmit downstream information signals onto the lines101, and each home 102 may have a receiver used to receive and processthose signals.

There may be one line 101 originating from the central office 103, andit may be split a number of times to distribute the signal to varioushomes 102 in the vicinity (which may be many miles) of the centraloffice 103. Although the term home is used by way of example, locations102 may be any type of user premises, such as businesses, institutions,etc. The lines 101 may include components not illustrated, such assplitters, filters, amplifiers, etc. to help convey the signal clearly,but in general each split introduces a bit of signal degradation.Portions of the lines 101 may also be implemented with fiber-opticcable, while other portions may be implemented with coaxial cable, otherlines, or wireless communication paths. By running fiber optic cablealong some portions, for example, signal degradation in those portionsmay be significantly minimized, allowing a single central office 103 toreach even farther with its network of lines 101 than before.

The central office 103 may include a termination system (TS) 104, suchas a cable modem termination system (CMTS), which may be a computingdevice configured to manage communications between devices on thenetwork of lines 101 and backend devices such as servers 105-107 (to bediscussed further below). The TS may be as specified in a standard, suchas, in an example of an HFC-type network, the Data Over Cable ServiceInterface Specification (DOCSIS) standard, published by Cable TelevisionLaboratories, Inc. (a.k.a. CableLabs), or it may be a similar ormodified device instead. The TS may be configured to place data on oneor more downstream channels or frequencies to be received by devices,such as modems at the various homes 102, and to receive upstreamcommunications from those modems on one or more upstream frequencies.The central office 103 may also include one or more network interfaces108, which can permit the central office 103 to communicate with variousother external networks 109. These networks 109 may include, forexample, networks of Internet Protocol devices, telephone networks,cellular telephone networks, fiber optic networks, local wirelessnetworks (e.g., WiMAX), satellite networks, and any other desirednetwork, and the interface 108 may include the corresponding circuitryneeded to communicate on the network 109, and to other devices on thenetwork such as a cellular telephone network and its corresponding cellphones, or other network devices. For example, the network 109 maycommunicate with one or more content sources, such as multicast orunicast video sources 110 a-b, which can supply video streams forultimate consumption by the various client devices in the homes 102.

As noted above, the central office 103 may include a variety of servers105-107 that may be configured to perform various functions. Forexample, the central office 103 may include a push notification server105 that can generate push notifications to deliver data and/or commandsto the various homes 102 in the network (or more specifically, to thedevices in the homes 102 that are configured to detect suchnotifications). The central office 103 may also include a content server106 configured to provide content to users in the homes. This contentmay be, for example, video on demand movies, television programs, songs,text listings, etc. The content server may include software to validateuser identities and entitlements, locate and retrieve requested content,encrypt the content, and initiate delivery (e.g., streaming) of thecontent to the requesting user and/or device.

The system may also include computing devices such as an image capturemanagement server 105 and one or more image capture servers 106 a . . .n. As will be described below, the image capture servers 106 may processmultiple video streams and periodically extract images from each stream.The image capture management server 105 may coordinate the image captureprocess and interact with various cache servers 107 to handle clientrequests for images and related services.

The central office 103 may also include one or more application servers107. An application server 107 may be a computing device configured tooffer any desired service, and may run various languages and operatingsystems (e.g., servlets and JSP pages running on Tomcat/MySQL, OSX, BSD,Ubuntu, Redhat, HTML5, JavaScript, AJAX and COMET). For example, anapplication server 107 may be used to implement a cache server for theimage capture system described herein. As will be described furtherbelow, the image cache server 107 may receive and process requests fromvarious clients for up-to-date images that have been captured by thesystem described herein. Other example application servers may beresponsible for collecting data such as television program listingsinformation and generating a data download for electronic program guidelistings. Another application server may be responsible for monitoringuser viewing habits and collecting that information for use in selectingadvertisements. Another application server may be responsible forformatting and inserting advertisements in a video stream beingtransmitted to the homes 102. And as will be discussed in greater detailbelow, another application server may be responsible for receiving userremote control commands, and processing them to provide an intelligentremote control experience.

An example home 102 a may include an interface 120. The interface 120may comprise a device 111, such as a modem, which may includetransmitters and receivers used to communicate on the lines 101 and withthe central office 103. The device 110 may be, for example, a coaxialcable modem (for coaxial cable lines 101), a fiber interface node (forfiber optic lines 101), or any other desired device having similarfunctionality. The device 111 may be connected to, or be a part of, agateway interface device 112. The gateway interface device 112 may be acomputing device that communicates with the device 111 to allow one ormore other devices in the home to communicate with the central office103 and other devices beyond the central office. The gateway 112 may bea set-top box (STB), digital video recorder (DVR), computer server, orany other desired computing device. The gateway 112 may also include(not shown) local network interfaces to provide communication signals todevices in the home, such as televisions 113, additional STBs 114,personal computers 115, laptop computers 116, wireless devices 117(wireless laptops and netbooks, mobile phones, mobile televisions,personal digital assistants (PDA), etc.), and any other desired devices.Examples of the local network interfaces include Multimedia Over CoaxAlliance (MoCA) interfaces, Ethernet interfaces, universal serial bus(USB) interfaces, wireless interfaces (e.g., IEEE 802.11), Bluetoothinterfaces, and others. Any of the devices in the home, such as thegateway 112, STB 114, computer 115, etc., can include an applicationsoftware client that can make use of the video images captured by theimage capture servers.

FIG. 2 illustrates general hardware elements that can be used toimplement any of the various computing devices and/or software discussedherein. The computing device 200 may include one or more processors 201,which may execute instructions of a computer program to perform any ofthe features described herein. The instructions may be stored in anytype of computer-readable medium or memory, to configure the operationof the processor 201. For example, instructions may be stored in aread-only memory (ROM) 202, random access memory (RAM) 203, hard drive,removable media 204, such as a Universal Serial Bus (USB) drive, compactdisk (CD) or digital versatile disk (DVD), floppy disk drive, or anyother desired electronic storage medium. Instructions may also be storedin an attached (or internal) hard drive 205. The computing device 200may include one or more output devices, such as a display 206 (or anexternal television), and may include one or more output devicecontrollers 207, such as a video processor. There may also be one ormore user input devices 208, such as a remote control, keyboard, mouse,touch screen, microphone, etc. The computing device 200 may also includeone or more network interfaces, such as input/output circuits 209 (suchas a network card) to communicate with an external network 210. Thenetwork interface may be a wired interface, wireless interface, or acombination of the two. In some embodiments, the interface 209 mayinclude a modem (e.g., a cable modem), and network 210 may include thecommunication lines 101 discussed above, the external network 109, anin-home network, a provider's wireless, coaxial, fiber, or hybridfiber/coaxial distribution system (e.g., a DOCSIS network), or any otherdesired network.

Various features described herein offer improved remote controlfunctionality to users accessing content from the central office 103 oranother content storage facility or location. For example, one such usermay be a viewer who is watching a television program being transmittedfrom the central office 103. In some embodiments, the user may be ableto control his/her viewing experience (e.g., changing channels,adjusting volume, viewing a program guide, etc.) using any networkeddevice, such as a cellular telephone, personal computer, personal dataassistant (PDA), netbook computer, etc., aside from (or in addition to)the traditional infrared remote control that may have been suppliedtogether with a television or STB.

FIG. 3 illustrates an example process that can be executed by softwareand components described herein, including computing devices such as animage capture management server 105, capture server 106, or cache server107. The following steps will be described as being performed byparticular servers, for the sake of simplicity, but any computingdevices can be used. In step 301, the management server 105 maydetermine whether any new clients have requested to access or registerfor the image sampling service and obtain security credentials (e.g.,password, keys, etc.) that may be used to later request and receivecaptured images. The clients can be any software and/or hardware deviceor process that wishes to obtain the images sampled by the image captureservers. For example, one client may be a digital video recorder (DVR)device running an application (e.g., an EPG-type application) thatoffers users images or other selective data from listed content, such asvideo programs. Another client may be an Internet server hosting a webpage that includes current or real-time images from one or more videoservices.

The registration itself can be requested in a variety of ways. Forexample, the clients may be provided with a URL to log in to, orregister for, the image capture service. The client may send an HTTPrequest to the image capture management server 105.

In response to a registration request, the management server 105 mayauthenticate the client in step 302. The authentication can take anydesired form. For example, a client's request may include a uniqueidentifier (e.g., an account login, a device address such as a mediaaccess control (MAC) address, etc.) and security information (e.g., adigital signature, password, etc.), and the management server 105 mayuse this information to determine if the particular client is permittedto access the image capture service. To make this determination, themanagement server 105 may issue a request to an entitlement server (notshown), identifying the user who may be associated with the client, andmay receive a response indicating what video services the user (or theuser's devices) is allowed to access. For example, some services mayrequire subscription and/or payment to access (e.g., pay services likeHBO, Showtime, or video on demand content), and the entitlement server'sresponse can identify what services the user is permitted to receive.The management server 105 may store information identifying the user'saccess rights (e.g. identifying permitted and prohibited services), andcan respond to the client and provide it with an additional key orpassword or other information that can be used for subsequentcommunications with the client. In some embodiments, this key orpassword can have a limited life (e.g., 10 minutes), and the client mayneed to issue a new login request periodically in order to continue toreceive access to the captured images described herein.

In some embodiments, the management server 105 is preconfigured with alisting of all content, such as video and/or data streams, that it needsto support. Alternatively, the server 105 may allow for the dynamicaddition/removal of supported streams (or other content). For example,the server 105 may automatically scan through one or more predeterminedaddresses known to be used for multicasting content, or a video assetmanagement system (e.g., a video on demand server) can affirmativelynotify the server 105 when a new stream begins. For such systems, instep 303, the management server may determine whether a new video ordata stream is being transmitted and needs to be added to the service.The streams may correspond to linear (e.g., transmitted according to apredetermined schedule) or on demand video services, such as televisionnetworks (e.g., ESPN, HBO, etc.) or Internet Protocol streaming services(e.g., webcasts, online videos, etc.) that are transmitting (e.g.,broadcasting, multicasting, or otherwise making available) content, andthis determination can be done in a variety of ways. For example, acontent server 106 at the central office 103 may be responsible foroffering the streams to its users, and in so doing, the content servermay be configured to send a message to the image capture managementserver 105, informing the management server 105 of the new stream. Themessage may identify, for example, details about the stream, such asinformation identifying the program's name, its service provider, itsvideo resolution, its encoding type and decoding requirements, an MPEG2transport stream identifier, and any other information needed to locatethe stream. A stream may be active so long as it is activelytransmitting content, such as streaming live video captured from acamera at a sporting event, or streaming a previously-recorded video inresponse to a video on demand request.

If a new stream is to be added, then in step 304 the management server105 may determine what capture parameters should be used for the newstream. The capture parameters may identify, for example, how frequentlyframes should be sampled from the stream (e.g., once per second), whatsize those frames should be (e.g., pixel resolution), color quality, andwhere the frames should be stored (e.g., creating a directory path orfile location on the server 105's memory device). In some embodiments,the capture parameters may be constant across all captured streams. Inother embodiments, different streams may be subject to differentparameters. For example, some programs may have their images capturedmore or less frequently than others. The images captured may also bemanipulated in different ways. For example, an advertisement image orwatermark logo may be added to the captured still image. The variousparameters may be established by the management server 105, by a contentprovider, by a requesting client device, or any other desired source.

In step 305, the management server 105 may assign the new stream to acapture server 106 a . . . n. The assignment may be based on the captureserver's existing load, as well as the parameters determined in step304, in order to balance the processing load among the capture servers.When a capture server is assigned a stream, the corresponding captureserver may begin a new instance of a software process to capture imagesfrom that stream (e.g., each stream being captured may be serviced byits own instance of image capturing software). The software instance maythen join or subscribe to the video stream. For example, the videostream may be a unicast feed, multicast feed, or source-specificmulticast feed, and the software instance may transmit a request to amulticast server to join the feed and begin receiving its video. Fromthere, the capture server 106 a . . . n can monitor the video stream,capture images at the rate and type specified in the capture parameters,generate a still image in accordance with those parameters (e.g., havingthe required pixel dimensions/size), and store the image in thespecified path or file location on the server 106's hard drive.Alternatively, the captured images may be transferred to a differentstorage location, such as the server 105's hard drive, or any othershared memory space.

The joining of a stream and capturing from the stream can be done in anydesired manner. For example, the capture server 106 a . . . n may joinan MPEG2 source-specific multicast stream, and may use a media tool suchas a VideoLAN client, MPlayer media player, or out of browserSilverlight application to capture the desired still images from thestream. For the MPlayer media player, an example command lineinstruction may be as follows:

mplayer -lavdopts skipframe=nonref:skiploopfilter=nonkey:lowres=2-sstep60 -nodouble -nosound -vc $codec -ni -vo jpeg:outdir=output-cnn-frames 1000 -vf scale=320:180, framestep=8 -reuse-socket -tsprog $3udp://@232.47.254.0:47000>/dev/null

The “-nosound” parameter may cause the player to omit processing ofsound data, which can be useful to streamline processing. The “lowres=2”parameter may establish a resolution or image quality level for thecapture of the images, while the “scale=320:180” parameter may set thepixel resolution to be 320×180.

The “-vc $codec” parameter may identify the codec being used to encodeand decode the stream.

The “jpeg:outdir=output-cnn” parameter identifies the file or pathlocation on the capture server 106 where the captured still imagesshould be stored.

The “-frames 1000”, “-framestep=8” and “-sstep60” parameters may definea limit on the number of frames to capture, or how frequently to capturethem.

The “-lavdopts skipframe=nonref:skiploopfilter=nonkey” parameter maycause the player to skip frames and relieve demands for CPU performance.

The “-reuse-socket” parameter may be useful if multiple instances of theMPlayer need to connect to the same IP port of a multicast feed if thefeed is using a multiple program transport stream (MPTS) format.

The “-tsprg <id>” parameter may help identify the particular program inthe transport stream, if the feed is using an MPTS. The “udp://”parameter may identify the multicast source IP address for the stream.

A call to a VideoLAN client can contain similar parameters identifyingthe output format, lack of audio, output location, etc., such asfollows:

cvlc -V image udp://@232.47.254.0:47000 -image-out-prefix=PREFIX-image-out-format=png -image-out-ratio=30 -image-out-width=320-image-out-height=180 -no-video-title-show noaudio

Other parameters may also be used to skip frames (ffmpeg-skip-frame orffmpeg-skip-idct), lower the resolution (ffmpeg-lowres, ffmpeg-fast,scene-ratio), use hardware decoding (ffmpeg-hw), and any other desiredoperation parameter.

The image capture server can also dynamically resize the images that areinitially captured, to provide one or more alternate sizes of the sameimage that can be made available. For example, after images are capturedby the VideoLAN client, the capture server 106 may pass the images toanother tool, such as ImageMagick, to generate resized versions.

Referring back to FIG. 3, after the video stream has been assigned to acapture server 106, the management server 105 may proceed to step 306,and add the new stream to a list file identifying the streams that arebeing actively captured. The list may be a simple text file stored onthe management server 105's hard drive, and may list the streams byservice provider or source name, source address (e.g., IP address),multicast port ID, transport stream ID, and universal resource locator(URL) identifying where the captured images for a particular stream areavailable. The list may also identify other parameters for the capture,such as time/date of latest or first capture, capture frequency, imagequality (size, resolution, pixel dimensions, compression type, etc.),title of current program being captured, alternate images captured, etc.

As noted above, some embodiments of the management server 105 may bepreconfigured to process a predetermined set of video streams, and neednot include the option of adding a new stream. Steps 304-306 may also beperformed in those embodiments, for example, when the serviceinitializes.

Returning to step 307, the management server 105 may determine if it hasreceived a request for captured images. For example, a client EPG on auser's computer 115 may be configured to incorporate the latest imagesfrom program sources in a channel listing, and may send the request forthe images. The request itself may be in any desired format. Forexample, the request can be an HTTP request such as the following:

http://flab.xcal.tv/api/request

In some embodiments, the request can be directed to the cache server107, and the cache server 107 can be configured to serve as atransparent proxy for the management server 105. The cache server 107can then redirect the request to the management server 105. Such atransparent proxy may allow the management server 105 and/or captureservers 106 to remain hidden from normal public view, as a securityprecaution. The cache servers 107 may instead be trusted with the actualURLs of the management server 105 and capture servers 106, and can storean address conversion or lookup table mapping the URL requested by theclient and the corresponding address/URL for the servers 105/106.

The request can also include parameters identifying the requestedinformation. For example, the request can be a request to obtain alisting of available services, or to retrieve captured images. In someembodiments, the request can specify which streams are of interest forsampling or viewing (e.g., allowing clients to avoid receiving imagesfrom streams that are of no interest to the client's user). In someembodiments, the request can ask to filter the still images to avoidreceiving images from certain types of programs or service providers.For example, the requesting client can ask that it not be given stillimages from adult video content, specific programs, or content fromcertain predefined service providers known to offer content in which theclient's user has no interest.

If a request has been received, then in step 308, the management server105 may authenticate the request. The authentication may be based on theauthentication performed when the client originally registered for theservice, and may involve a simple lookup to determine whether theclient's identifier (e.g., its MAC address) is found in a listing ofauthorized clients maintained by the server 105 from step 302'sauthentication. Alternatively, the request may carry with it a valuethat has been signed by a key provided in the authentication 302, or apassword, or any other desired form of authentication that can determineif the requesting client is permitted to receive the requested content.As another alternative, the request can simply identify the requestinguser or device (e.g., a subscriber ID, a hardware identifier, anaddress, etc.), and the management server 105 can use thatidentification information to determine the preferences and/orentitlements of the user or device. The management server 105 can alsoprovide the identification information to an entitlement server (notshown), requesting confirmation of the user'sentitlements/restrictions/preferences.

If the request is authenticated, then the server 105 may proceed to step309, and collect the relevant information that is responsive to therequest. This may include examining the request, or querying therequesting client, for parameters and/or restrictions on the informationrequested. For example, the requesting client may provide a list ofcontent, such as television programs or video content services (e.g.,ESPN, HBO), whose still images or other descriptive data are requested,or whose still images are to be avoided. The requesting client can alsospecify formatting preferences for the requested images, such as desiredsizes, file formats, etc. In some embodiments, the management server 105and/or capture server 106 can be configured to dynamically resize imagesfor requesting clients, in response to a client's request. Additionally,or alternatively, the capture servers can be configured to capture andstore multiple different versions of images from the stream, storingthem in different sizes, formats, etc. The server 105 may then gatherthe information that is responsive to the request, and return it to therequesting client. The response can include data such as a series ofrecords, one for each stream monitored by the system and available tothe requesting user, and the following is an example of a record thatcan be returned to the client:

{“name”:“18324”,“source”:“10.252.251.74”,“active”:true,“node”:“http://ia.ula.lab.xcal.tv”,“imageUrl”:“http://127.0.0.1:8080/liveimage/18324”,“multicastlp”:“232.39.254.231”,“multicastPort”:“39238”,“tsProgramld”:“1”,“codec”:“ffh264”,“cdnUrl”:“ccp-cmc-smooth-pod01iis01.cds.bdn.lab.xcal.tv/cnn.isml/manifest”,“longName”:“CNHNHD”,“stationld”:“8537732268987025117}

The above example is an entry for a single feed being captured by thecapture service. The parameters returned can include a “name” parameter(18324) as a handle identifying this particular record; a “source”parameter (10.252.251.74) identifying a multicast source IP for thestream; and an “active” parameter (true) indicating whether thisparticular stream is actively being captured. The “active” parameter maybe used, for example, to indicate whether a recent capture for thestream has been successfully captured. If the stream has experienced anerror or is no longer transmitting, or if an error capturing the imagehas occurred, the “active” value can be set to “false,” to indicate thatthere is no current valid image available for the stream, but that thestream remains among the ones that the system is attempting to capture.

The “node” parameter (http://ia.ula.lab.xcal.tv) can identify the URL ofthe specific capture server 106 that has been assigned to capture thisstream. In embodiments in which the cache server 107 acts as the pointof contact for clients, a well-behaved client will have no need for thedirect “node” URL, as the client will be interacting with the cache 107,and not the actual server 106.

The “imageURL” parameter (http://127.0.0.1:8080/liveimage/18324)provides the URL that the requesting client should use to retrieve thecurrent image for this stream. This URL can be registered to point tothe cache 107, and the cache 107 can contain its own internal lookuptable to redirect the request to an address of the management server 105where the current image can be found.

The “multicastlP” parameter (232.39.254.231) and tsProgramID (1)parameters can identify the multicast group IP address and transportstream identifier for the stream being captured.

The “codec” parameter (ffh264) may identify the codec (coding/decoding)being used for the stream, and the “cdnURL” (ccp-cmc-smooth-pod01 . . ./manifest) may provide the URL for the video stream being captured. Thismay be used, for example, if the client wishes to directly view thestream.

The “longName” parameter (CNHNHD) may be an alphanumeric or textual nameof the stream represented by the record.

The “stationid” parameter (85377322 . . . ) may be an identifier used toassociate the current record with an overall metadata EPG database. Forexample, if the current program being streamed on the CNHNHD network isa nightly news program, then the stationid parameter may correspond toan identifier used to identify that program in an EPG database, whichcan contain additional information about the program (e.g., its nightlytopics, text description, duration, other air times, etc.). This valuemay be useful, for example, if a client wishes to request additionalinformation about the given program, in which case the client can usethis identifier to submit a request to the EPG database.

In addition to retrieving and returning entries identifying the streamsbeing captured, the manager 105 can respond to other types of requestsas well. For example, a client may request to receive a slate of themost recent image (or predetermined number of images) for a plurality ofservices identified in the request. In response, the cache 107 (via themanager 105) may provide the URLs for the images in the sample entryabove, and the client may issue URL requests for the particular imagesof interest to the client.

In step 310, the responsive information may be transmitted to the clientfor use in the client's own application. At that point, the client maydetermine which of the listed records is of interest, and can issue newrequests to the corresponding URLs.

In step 308, if the client's request fails authentication, then theserver 107 may respond in step 311 by reporting the denial to the client(or outright ignoring the request).

Proceeding to step 312 (FIG. 3 b), the manager server 105 can check todetermine if there has been a request to change any of the capturingparameters. As noted above in step 304, each stream can be capturedaccording to its own set of parameters (e.g., its own capture rate,image quality, etc.) that can be established when capturing begins.After capturing has begun, however, the manager server 105 and/or videosource (e.g., source 110 a) may be asked to change those parameters. Forexample, the manager server 105 may receive indication from a captureserver 106 that the server 106 has exceeded a maximum processingthreshold, or is exhausting processing resources, and the server 105 maydetermine to reduce the workload by adjusting the capture parameters orreallocating the capturing to a different capture server 106. As anotherexample, a new client may register with the system, and may make arequest to receive a different type of still image (e.g., differentresolution, different capture frequency, etc.), and the manager server105 may determine to alter the capture parameters to accommodate the newclient's request.

In step 313, the manager 105 may send an instruction to one or more ofthe capture servers 106 a . . . n, instructing the servers to changetheir capturing parameters. The capture servers 106 a . . . n may acceptthe new parameters and restart the various capture instances for thevarious streams or feeds that they are capturing.

In step 314, the manager 105 may determine if a stream has ended. Forexample, if a program being streamed terminates, or is no longer beingviewed by viewers, the stream may be torn down by the streaming serveror video source 110, and accordingly, the capturing for those streamscan be torn down as well. The streaming servers may be configured totransmit a message to the manager 105, informing if of the terminationof the stream. Alternatively, the manager 105 may periodically (e.g.,once per minute) inspect all of the streams in the active stream list,and check to verify that those streams remain active (e.g., by lookingfor recent images, or by sending a heartbeat query to the multicastserver handling the stream).

If a stream has ended or is no longer desired or available, then themanager 105 may proceed to step 315, and instruct the correspondingcapture server to terminate its capturing of the stream, and to releaseany resources that were reserved and/or occupied by the capturingprocess for that stream. In step 316, the manager 105 may update itsactive stream list to remove the entry for the closed stream, or tootherwise indicate that the stream is no longer being captured.

In step 317, the manager 105 (or other computing devices such as thevarious capturing servers 106) may run a cleaner routine. The cleanerroutine may inspect the file directories in which each capturer instance(e.g., each instance of the capture software that is running on aserver) is storing images, and delete outdated or unneeded images. Forexample, the cleaner may be configured to maintain a predeterminednumber (e.g., 5, 10, 100, etc.) of previous images for each stream, andcan delete the older images that are no longer needed. Alternatively,each captured image may be associated with a time to live value, and thecleaner routine may involve determining whether an image's time to livehas expired. For example, an image may have a time of 10 seconds to livein the capture directory, and 10 seconds after the image is captured,the image will be deleted from the directory (or the next time thecleaner runs after 10 seconds have passed). In some alternativeembodiments, each capture instance may maintain a predetermined numberof images (e.g., 5), each having a time to live, and the captureinstance can capture a new image only if one of the existing images hasexceeded its time to live.

In step 318, the manager 105 may monitor system data and requests, suchas by running a watchdog routine. The watchdog routine may monitor thevarious active streams that are identified in the active stream list,and verify that new files are being written to the directory accordingto the requirements set in the list. For example, if the capture server106 a is supposed to be capturing one still image per second for aparticular stream, the watchdog routine may inspect the directory overthe course of several seconds to confirm that new images are indeedbeing written. This can include, if desired, a comparison with priorimages to confirm that the image content is actually changing as well.Additionally, the watchdog may inspect the image files themselves, toconfirm that the files are valid image files (e.g., such as by using theUnix file command) and are not corrupted in some fashion. If thewatchdog identifies a directory in which image files are no longer beingwritten, or the images are not being updated, the manager server 105 cantransmit an inquiry to the corresponding capture server 106 to requestits current status for the capture instance that is responsible forcapturing the given stream, and to take corrective action (e.g.,restarting the instance, tearing it down due to a closed stream, etc.).

The process can then return to step 301, and repeat indefinitely (oruntil the capture service is paused, terminated, or no longer desired bya client).

As a result of the FIG. 3 A&B process or similar processes, clients maybe able to obtain current, real-time, image captures for various videostreams. One use of the disclosed methods may be applicable to guides,such as electronic program guides (EPGs). Traditionally, EPGs haveoffered users a textual listing, typically in a grid format, oftelevision shows arranged by channel and time. While the text grid EPGremains a mainstay for some viewers, there is an increasing need forsimpler and more appealing ways to search for and access content. FIGS.4 A&B illustrate several examples of the types of features that can beimplemented by using the capture service, which can, for example, offerusers a more intuitive way to find and view content. In FIG. 4A, anonscreen mosaic 401 can be displayed on a consumption device (e.g., aDVR's associated display, a computer, a smartphone display, etc.). Tosupport this, for example, the consumption device can simply register(as in step 301), and then transmit periodic requests to the manager105, requesting the latest images that have been captured for eachservice.

The client device may display the images (or other data descriptive ofthe content) in the mosaic 401, along with source identificationinformation (e.g., “HBO,” “CBS”, etc.), and allow the user to peruse themosaic to select a program. While the user is viewing the mosaic 401,the client can repeatedly request updated images from the capturemanager 105, and update the images appearing in the mosaic 401. In thismanner, the user viewing the mosaic 401 can see a dynamically updatingset of tiles reflecting the current happenings on that piece of contentor service. This can allow the user to identify a program (or portion)that may be of the most interest, and choose it for viewing and/orrecording. The user can navigate through the mosaic 401 by moving acursor to highlight different tiles, and the highlighted tile may beenlarged with respect to other tiles. When the user chooses a tile forviewing, the client can then send a message to the server supportingthat service, and request to receive the content, such as join themulticast group for that service.

FIG. 4B illustrates an alternative heat map mosaic 402. In thisalternative, the tiles for the different programs may be sizeddifferently, according to how popular or highly rated the program is.For example, a more popular program may be shown in a larger tile, withmore resolution, than other less popular programs. The popularity of aprogram can be determined in any desired manner. For example, if theprograms are all multicast feeds, then the client (or the manager 105)may request information from various multicast servers to identify thefeeds that have the most receiving users. To assemble the heat map, theclient may request differently-sized versions of the different services,or the manager 105 can provide the different sizes instead (if themanager 105 is handling the popularity determination). This example heatmap uses size to differentiate the more popular programs from the lesspopular ones, but other approaches may be used as well. For example,more popular programs may be given a brighter appearance or lightercolors. Another alternative would be to use positioning to indicate themost popular program. For example, the most popular program may bepositioned in the center of the display, with less popular programsappearing towards the outer periphery. As a further example, differentheat maps can be generated for different categories of programs. Forexample, different heat maps can be generated for dramas and newsprograms. FIGS. 4A & 4B illustrate example display interfaces, but anyother desired display of the captured images may be used.

In some embodiments, a user may select one of the images to filter theview to depict programs that are similar to the selected one, therebyallowing for a recommendation based on similarity. The similarity can bebased on any desired measure, such as genre, actors, series (episodiccontent, movie sequels, etc.), etc. The recommended content can bedifferentiated from the other content in many ways, such as the heat mapexamples discussed above. In other examples, the less popular programsmay be removed altogether from the display, or their images may nolonger be updated (e.g., resulting in only the recommended programshaving the updated images). The selection may be made, for example, bynavigating a cursor or mouse pointer and clicking on an image. Anintermediate selection may be made as well by hovering a pointer over animage.

In some embodiments, a user's selection of a program may be transmittedto the management server 105, which can then aggregate the selection todetermine which programs are the most popular. A user's selection of aparticular program can also be forwarded to other individuals, such asthose who have previously identified themselves to be friends with theselecting user. The selection and/or associated images can also bepassed along to social media servers for further user and/ordistribution.

FIG. 5 illustrates another example implementation of the disclosure,involving a live channel preview for an EPG 501. In the EPG 501, theuser may be permitted to move a cursor or highlight through a grid ofcells that identify program titles, and as the user highlights eachcell, the client EPG 501 may repeatedly request updated still images forthe highlighted program to give the user a live preview of what is beingshown on the channel. So, if the user highlights the “Dear Food Network”program in the grid, the EPG can dynamically request and display apreview window 502 for that program, where the preview window 502 isupdated once every five seconds to show the latest image from that show.This preview may be useful, for example, if the user does not wish totune to the new show when it is during a commercial break. Concurrently,the user may also be provided with a currently-tuned window 503, showingthe live images from the show that the user was previously watchingbefore entering the EPG.

Another client preview implementation may involve channel changes. Whenthe viewer requests to change the channel or switch to viewing adifferent stream, it can often take a moment for the user device's(e.g., STB's) decoder to locate, tune and decode the stream carrying thenew program or service. In some embodiments, when a channel changerequest is received, the user device may issue a request to retrieve themost recent image for the new channel, and can display that last imagewhile the user device obtains the live feed for that new channel.Alternatively, the user device can request a predetermined number of thecaptured images (e.g., the last 5 images), and can present those in atimed sequence prior to displaying the current stream. In that manner,the user can be given a preview and advanced context for what ishappening in the program, and perceived “down time” can be minimized Insome embodiments, when a user tunes to a particular channel or service,the client device can automatically request and retrieve captured imagesfor one or more neighboring channels or services (e.g., the next threechannels or services that would be displayed should the user press achannel up or channel down key on a remote control three times). Thesecaptured images may be stored for use should the user press the channelup/down key, providing the user with immediate feedback on the channelchange.

Another client implementation can involve a user who wishes to view adifferent portion of the program they are currently watching. The usercan be presented with a slider bar representing the timeline of thecurrent program, and can be given the option of dragging a pointer alongthe timeline to select a position to select a starting point for playingback the program content. As the user drags the pointer along thetimeline, the client device can retrieve and display the still imagenearest the pointer's position along the timeline. For example, theclient device can retrieve and cache still images for content that theuser is viewing (and also for content the user is not viewing), andstore those still images along with a time identifier indicating thetime within the program represented by the still image. As the usermoves the pointer to make a selection, the client device can determinethe time of the pointer relative to the start of the program, find astill image that is nearest to that time, and display that image toassist the user in selecting a starting point.

In some embodiments, the user may be given the option to rewind and/orfast forward through content appearing in either of the windows 502/503.For example, the user can move a cursor through the grid guide tohighlight a particular program, and a preview of the program may appearin window 502. The user may then enter a rewind or fast-forward command,and the client device can map the command to the program highlighted bythe cursor, and can retrieve earlier or later images from the capturedimages and present them to the user in window 502, thereby allowing theuser to fast forward or rewind through a program that is being previewedin the guide.

FIGS. 6 and 7 illustrate screens or interfaces for anotherimplementation of the disclosure, which can be made at the managerserver 105, for example, to assist in managing which stream captureswill be made available to requesting clients. In the FIG. 6implementation, the user (which can be an administrator of themanagement server 105) may request a listing of the current streamsbeing captured by the capture servers 106 a . . . n, and viewinformation for each from the manager 105's active stream list. Thedisplayed information can include the textual name of the service, theIP address of the source's server providing the video stream, addressand port information for the particular stream, transport streamidentifiers for streams that may be carried in a multiple programtransport stream, and a URL for the latest image captured from thatstream. Other information about the captures (e.g., the number and typesof versions captured, the capture rate or frequency, the image filesizes, etc.) may be displayed as well, if desired.

Along with each entry, the FIG. 6 display can include an option toselect or deselect each entry. Selected streams may have their capturedimages made available to clients, while unselected streams might nothave their captured images available, and the images being madeavailable can be viewed on another mosaic 701. With the FIG. 6 & 7implementation, a system administrator at the manager 105 can inspectthe images being made available, and identify problems such as deadstreams. Also, the administrator can use the interface to define one ormore mosaic profiles, picking and choosing different services to beincluded in each profile. So, for example, the administrator can definea “football” profile, and select the services that are streaming livefootball games. A client can request that particular profile, andreceive captured still images for ongoing football games.

In some embodiments, the management server 105 may be configured toprevent end clients from identifying which capture server 106 iscapturing particular streams. To obscure that assignment, the managementserver 105 may be implemented as a web server with proxying through thecache server 107, such that client requests are directed to the client107, which in turn communicates with the management server 105.

As noted above, the management server 105 may be a standalone computingdevice, or it can be implemented as a process on the same device as oneof the capture servers 106. When implemented as a standalone, it canserve as an aggregation server, collecting information from the variouscapture servers regarding what is being captured. To collect thisinformation, the server 105 and capture servers 106 may use RESTfulAPI's to facilitate the exchange of information being captured. Theserver 105 can also serve as a proxy for incoming requests, anddistribute the requests on to the various capture servers 106 when, forexample, static images are needed.

In some embodiments, the management server 105 and capture server 106may be identical hardware, both configured to perform either role, andthe selection of one or the other role for a particular piece ofhardware can be made based on any desired factor, such as demand,estimated need, etc.

In some embodiments, the features above may be implemented using toolssuch as the Linux operating system, Tomcat and Varnish tools, and imagetools such as MPlayer, VLC, Silverlight and/or ImageMagick to performthe capturing and resizing. Nagios tools may be used to help monitor theperformance of the various system components.

Applicants have noted that some optimization parameters may be used inLinux. The Linux soft nofiles parameter may be increased (e.g., set to16384 files) to increase the number of soft files that a user ispermitted to have open at one time, while the Linux hard nofilesparameter may also be similarly increased to allow more open hard files.The queue length (e.g., net.unix.max_dgram_qlen=4096) may be increasedto allow for a larger packet queue, and a queue backlog timer (e.g.,net.core.netdev_max_backlog=40000) may be increased to allow for moretime to process queues.

Various memory values may be increased as well, to allocate more spacefor handling the processes described herein. The following are examplememory settings that can be used in a Linux implementation:

net.ipv4.udp_wmem_min=65536

net.ipv4.udp_rmem_min=65536

net.ipv4.udp_mem=33554432 33554432 33554432

net.core.rmemdefault=33554432

net.core.wmem default=33554432

net.core.rmem_max=33554432

net.core.wmem_max=4096

Ethernet flow control can also be deactivated, so that the manager 105can simply drop packets when it gets too far behind in its packethandling, without asking the capture servers 106 to adjust their captureand/or reporting rate. Also, the various security measures (e.g., use ofkeys, duration of keys, use of the cache server 107 to proxy access tothe management server 105 or capture servers 106) can be deactivated asdesired.

The various features described above are merely nonlimiting examples,and can be rearranged, combined, subdivided, omitted, and/or altered inany desired manner. For example, features of the servers can besubdivided among multiple processors and computing devices. The truescope of this patent should only be defined by the claims that follow.

1. A method, comprising: periodically capturing still images from eachof a plurality of content streams; dynamically updating a predeterminedstorage location associated with each content stream to contain a mostrecent captured still image from the corresponding content stream; andproviding a location address for each content stream's correspondingstorage location to one or more requesting clients.
 2. The method ofclaim 1, further comprising: storing, for each of the content streams,stream-specific parameters identifying a frequency with which stillimages are to be captured for the corresponding stream.
 3. The method ofclaim 1, further comprising: storing, for each of the content streams,stream-specific parameters identifying a size of still images are to becaptured for the corresponding stream.
 4. The method of claim 3, whereinthe parameters identify different sizes for different ones of thestreams.
 5. The method of claim 1, further comprising collecting andassembling the captured still images for a plurality of the contentstreams selected by a client.
 6. The method of claim 5, furthercomprising presenting an electronic program guide to a user, theelectronic program guide including a mosaic of current images fromlisted programs.
 7. The method of claim 1, further comprising runningdistinct instances of image sampling processes on a capture server foreach of the subscribed content streams.
 8. The method of claim 1,further comprising: storing, for each of the content streams,stream-specific parameters identifying a how still images are to becaptured for the corresponding stream; and adjusting the stream-specificparameters in response to a new client registration.
 9. The method ofclaim 1, further comprising: periodically scanning the location addressfor a first stream and determining a count of a number of captured stillimages; and in response to determining that the count exceeds athreshold limit, deleting one or more of the captured still images. 10.The method of claim 1, further comprising: periodically monitoring thelocation address for a first stream, and determining if an age for alast captured still exceeds an age limit; and in response to determiningthat the age limit has been exceeded, sending a request to restart acapturing process for the first stream.
 11. A method, comprising:obtaining network location address information identifying locations forcurrent captured still images from a plurality of active video streams;periodically retrieving current captured still images for each of theactive video streams; and generating an onscreen display wherein, foreach stream, a latest captured still image for the stream is included inthe display.
 12. The method of claim 11, wherein different videoprograms are represented in the program guide by different-sized currentstill images.
 13. The method of claim 12, wherein the size of the stillimages is based on the popularity of its corresponding video program.14. The method of claim 11, further comprising dynamically updating thedisplayed captured still images at a first time interval.
 15. The methodof claim 10, wherein each of the streams is a client-selected stream.16. The method of claim 1011 further comprising: determining apopularity for each program listed in the program guide; and positioningthe still images based on the popularity, wherein higher popularityprograms are positioned closer to a center of the display, and lowerpopularity programs are positioned farther from the center.
 17. Themethod of claim 11, further comprising: determining a popularity foreach program listed in the program guide; and using the popularitydetermination to alter an image capturing frequency for the program. 18.The method of claim 17, further comprising assigning lower capturefrequencies to programs having lower popularity.
 19. A method,comprising: displaying a schedule of available video programs;displaying a live stream of a currently-tuned video program in a firstwindow of the display; and displaying a most-recently captured stillimage from a second video program in a second window of the display. 20.The method of claim 19, further comprising updating the second window nomore than once per second.